Computerized Adaptive Testing in Instructional Settings

Item response theory (IRT) has most often been used in research on computerized adaptive testing (CAT). Depending on the model used, IRT requires between 200 and 1,000 examinees for estimating item parameters. Thus, it is not practical for instructional designers to develop their own CAT based on the IRT model. Frick improved Wald" s sequential probability ratio test (SPRT) by combining it with normative expert systems reasoning, referred to as an EXSPRT-based CAT. While previous studies were based on re-enactments from historical test data, the present study is the first to examine how well these adaptive methods function in a realtime testing situation. Results indicate that the EXSPRT-I significantly reduced test lengths and was highly accurate in predicting mastery. EXSPRT is apparently a viable and practical alternative to IRT for assessing mastery of instructional objectives. [] For many years, instructional developers and classroom teachers have used fixed-length, paper-and-pendl tests to assess student achievement of cognitive learning objectives. With the advent of interactive mainframe and minicomputers in the 1970s, it became possible to implement item response theory in the testing environment (Weiss & Kingsbury, 1984). Questions arose as how to best use this technology for computerized adaptive testing (CAT) and which decision algorithms to use. In order for readers who are not ~ with CATs to better understand the concept, the various forms of CAT are compared and contrasted here. Two examples are provided to illustrate how a CAT functions. Described in the last section of the article is a study that examined the efficiency and accuracy of the various CATs discussed. The goal of a CAT is to use the least amount of questions necessary to determine the level of performance of the examinee. Moreover, many CATs attempt to tailor a test to an individual's achievement level by avoiding questions that are extremely easy or difficult for a particular examinee and instead choosing questions that are closer to his or her ability level. It should be noted that several assumptions are made in order to use CATs appropriately. First, only one learning objective can be tested at a time. This objective can be as narrow or wide as desired, but only a unidimensional